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Executive  Summary 


The  goal  of  this  project  was  to  continue  to  investigate  methods  and  means  to  further  the  SERC 
Capstone  Marketplace  that  was  established  in  2013,  refined  in  2014,  and  further  developed  and 
designed  in  the  2016-2017  academic  year.  Previous  SERC  research  in  RT-19/19a,  RT-43,  RT  105, 
and  RT-131  has  shown  that  multidisciplinary  capstone  programs  can  enhance  development  of 
systems  engineering  competencies. 

The  Capstone  Marketplace  is  an  online  tool  intended  to  match  multi-disciplinary  student  teams 
with  challenging  engineering  projects.  While  web  based  tools  exist  for  matching  students  to 
projects  at  individual  institutions,  the  Capstone  Marketplace  is  intended  to  enable  broader 
participation  and  as  such  is  open  to  all  institutions  with  the  capability  of  participating  in 
undergraduate  level  capstone  projects.  The  marketplace  project  sponsors  and/or  mentors 
provide  domain  expertise  and  advice,  students  research  the  project  details  and  work  towards 
solutions,  while  faculty  supervisors  help  guide  the  teams  and  grade  their  work.  The  Capstone 
Marketplace  makes  it  easier  for  sponsors  to  reach  out  to  a  broad  pool  of  students  and  provide 
them  with  the  added  benefit  of  an  engaged  and  knowledgeable  mentor.  Students  can  more 
easily  find  projects  best  matched  to  their  interests  and  needs  and  faculty  have  an  open  source 
for  student  projects  that  can  lead  to  new  research  partners  and  collaboration. 

The  project  consisted  of  four  phases: 

1.  Initiating  and  supporting  a  fourth  and  fifth  year  of  Capstone  Marketplace  projects 

2.  Marketing  the  Capstone  Marketplace  to  attract  additional  sponsors  and  participating 
schools 

3.  Refining  the  Capstone  Marketplace  website/portal 

4.  Development  of  support  materials  for  sponsors  and  schools  that  use  the  marketplace 

Activities  of  these  phases  overlapped,  but  initiation  and  support  of  the  past  year  and  marketing 
for  future  projects  were  mostly  separate. 

Initiating  and  Supporting  Capstone  Marketplace 


Participating  Sponsors  and  Projects 

United  States  Special  Operations  Command  (USSOCOM)  was  the  main  sponsor  for  the 
academic  years  2015-2016,  with  the  2016-2017  extension,  with  several  capstone  projects 
available  for  students  and  faculty  to  choose.  The  original  projects  listed  for  the  academic  years 
were  as  follows.  As  it  will  be  mentioned  later,  there  was  then  potential  for  the  faculty  advisors 
and  students  to  submit  proposals  for  USSOCOM's  consideration. 


Table  1.  SERC  Sponsored  Projects  Listed  Online 


Sponsor 

Project 

*SOCOM 

Advanced  Body  Armor 
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*SOCOM 

Austere  Landing  Zone  Assessment 

*SOCOM 

Avoiding  Information  Overload 

*SOCOM 

Vessel  Disablement 

*SOCOM 

Enhanced  Performance  #6 

*SOCOM 

Water/Pressure-Activated  Personal  Flotation  Device 

SOCOM 

Armored  Window  Improvement 

SOCOM 

Body  Worn  Processor 

SOCOM 

Novel/Innovative  Communications 

SOCOM 

Communications  Junction  Box  Size  Reduction 

*SOCOM 

Advanced  Power  Technologies 

Asterisk  denotes  projects  where  there  were  identified  potential  participants. 


Participating  Schools/Proposed  Projects 

The  SERC  invited  engineering  schools  to  participate  in  the  2015-2016  academic  year  prior  to 
October.  This  resulted  in  five  schools  being  able  to  participate  -  5  faculty  advisors  with  a  total 
of  23  students. 


Table  2.  SERC  Projects  Selected  for  Funding 


Project  Name 

School 

Academic  POC 

Enhanced  Performance  (#6)  (Team  EPITOME) 

Stevens  Institute  of  Technology 

Dr.  Vikki  Hazelwood,  5  students 

Enhanced  Performance  (#2)  (Team  Exoskeleton) 

Stevens  Institute  of  Technology 

Kishore  Pochiraju,  4  students 

Water/Pressure-Activated  Personal  Flotation 
Device  (cont  #5)  (Team  BloFish) 

Stevens  Institute  of  Technology 

Dr.  Marissa  Gray,  4  students 

Austere  Landing  Zone  Assessment  (#7  Cont) 
(Team  Austere  Landing  Zone) 

Stevens  Institute  of  Technology 

Dr.  Leslie  Brunell,  4  students 

Underwater  Navigation  and  Communications 
(Team 

Georgia  Tech 

Dr  Mick  West,  6  students 

Shortly  after  the  2016  academic  came  to  a  close,  the  sponsors,  specifically  Commander  Greg 
Sutton,  requested  that  the  project  continue  into  the  summer  to  make  further  improvements  to 
the  design  after  a  visit  to  Virginia  Beach  to  debrief  on  the  progress  of  the  project. 

In  academic  year  2016-2017,  several  projects  were  proposed  for  approval  to  USSOCOM.  This 
also  included  a  new  format  agreed  upon  between  SERC  and  USSOCOM.  After  reaching  out  and 
networking  to  several  participating  universities,  it  was  noted  that  there  were  numerous  ideas 
on  projects  that  may  be  of  particular  interest  to  the  sponsors.  On  the  website,  we  added  the 
feature  to  propose  a  project  to  the  sponsor.  In  the  new  feature,  we  listed  the  mission  and 
objective  statements  of  USSOCOM  and  included  their  website  for  the  faculty  and  students  at 
participating  universities  to  view  and  ensure  their  proposed  projects  were  in  line  with  the 
sponsor's  objectives.  We  also  met  and  discussed  the  individual  potential  projects  with  each 
faculty  member  to  help  them  better  understand  the  Capstone  Marketplace  and  working  on  a 
DoD  contract.  This  concluded  in  the  Precipitation  Driven  Flood  Forecast  Model  proposed 
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project  to  be  outside  USSOCOM's  scope,  further  solidifying  the  advantage  of  meeting  with  the 
faculty  member  beforehand  to  overview  the  FARs  and  DFARs  associated  with  working  on  this 
contract.  The  2016-2017  Proposed  Projects  were  as  follows: 


Table  3.  SERC  Proposed  Projects  for  2016-2017  Academic  Year 


Project  Name 

School 

Academic  POC 

Enhanced  Performance  (#6) 

Stevens  Institute  of  Technology 

Dr.  Vikki  Hazelwood 

Enhanced  Performance 

Stevens  Institute  of  Technology 

Dr.  Kishore  Pochiraju 

Water/Pressure-Activated  Personal 

Flotation  Device 

Stevens  Institute  of  Technology 

Dr.  Marissa  Gray 

NEW:  Precipitation  Driven  Flood  Forecast 
Model  (removed  after  individual  meeting) 

Stevens  Institute  of  Technology 

Dr.  Alan  Blumberg 

NEW:  Reduction  of  Slamming  Loads  on 
Small  High-Speed  Craft 

Stevens  Institute  of  Technology 

Dr  Raju  Datla 

Water/Pressure-Activated  Personal 

Flotation  Device 

Stevens  Institute  of  Technology 

Dr.  Mirjam  Furth 

Manned  Mini-submarine  Variable  Depth 
Anchoring  System 

Stevens  Institute  of  Technology 

Dr.  Milan  Simonovic 

NEW:  TBD/Fatigue  Testing  Applicable  to 
both  Humans  and  Animals  (Spring 
Semester) 

Stevens  Institute  of  Technology 

Dr.  Marissa  Gray 

Human  Performance  Augmentation  (Spring 
Semester) 

North  Carolina  A&T  State 

University 

Dr.  Tonya  L.  Smith-Jackson 

NEW:  Power  line  awareness  for  Low 

Altitude  Aircraft 

University  of  Alabama  Huntsville 

Dr.  Shankar  Mahalingam 

NEW:  Real-time  External  Perspective  for 

Drones 

University  of  Alabama  Huntsville 

Dr.  Shankar  Mahalingam 

Those  designated  as  "new"  are  projects  proposed  by  students  and/or  faculty  that  were  not 
already  listed  Capstone  Marketplace  projects  on  the  website  by  the  sponsor.  Unfortunately,  we 
were  unable  to  initiate  projects  due  to  delay  in  the  academic  year.  The  following  were  the  ones 
that  were  able  to  start.  Dr.  Marissa  Gray  continued  to  work  on  the  Enhanced  Performance  #6 
with  John  Anderson  from  USSOCOM. 


Table  4.  SERC  Proposals  Selected  for  2016-2017  Academic  Year 


Project  Name 

School 

Academic  POC 

Enhanced  Performance  (#6) 

Stevens  Institute  of  Technology 

Dr.  Vikki  Hazelwood 

Water/Pressure-Activated  Personal 

Flotation  Device 

Stevens  Institute  of  Technology 

Dr.  Marissa  Gray 

Water/Pressure-Activated  Personal 

Flotation  Device 

Stevens  Institute  of  Technology 

Dr.  Mirjam  Furth 

Along  with  the  above  tasks,  we  were  able  to  also  quickly  turnaround  a  proposal  from  Auburn 
University  titled  "Open  Source  Investigations  into  Three  DOD  Relevant  Research  and 
Technology  Questions"  with  Dr.  Robert  Norton. 
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Project  results 


Projects  that  had  points  of  contact  at  USSOCOM  were  deemed  successful,  with  continuations 
into  summer  or  the  following  academic  year.  Those  that  were  not  as  successful  had  several 
conditions  associated  that  may  have  contributed  to  the  decreased  performance,  such  as: 

•  Inability  to  establish  proper  initial  communications  between  sponsor  point  of  contact 
and  the  team. 

•  Rushed  timeframe  due  to  delay  in  approval  and/or  funding. 

The  first  point  is  a  recurring  problem  and  is  addressed  in  a  later  section  of  this  report.  The 
second  is  as  well.  There  have  been  unconventional  teams  in  previous  CM  projects  that  operated 
outside  of  the  typical  capstone  design  format  and  managed  great  success,  including  in  this 
academic  year.  However,  the  fault  likely  lies  with  the  inability  to  obtain  the  best  students  for 
the  projects  and  also  the  ability  to  execute  in  the  structured  format  that  most  faculty  advisors 
are  used  to. 


Integrated  Lifecycle  Model/Roadmap  Forward 

The  core  of  Capstone  Marketplace  is  the  website  which  establishes  where  students,  faculty,  and 
mentors  can  view  the  potential  projects  suggested  by  the  sponsor  as  well  as  submit  proposed 
projects.  Below  is  a  timeline  of  the  process  of  identifying,  posting  and  matching  projects  to 
successful  applicants  and  sponsor  subject  matter  experts  (SMEs).  The  website  was  populated 
with  10  of  12  projects  for  the  2015-17  academic  years  by  close  of  the  spring  semester, 
however,  a  firmer  foundation  on  the  management  and  expectations  of  the  CM  should  be 
established.  A  more  appropriate  time  frame  is  mid-spring  semester  when  students  begin  to  pre¬ 
select  projects  and  develop  relationships  that  lead  to  team  formation.  The  nature  of  the 
academic  calendar  and  its  deviation  from  the  primary  projects  sponsor's  standards  present  a 
distinct  challenge  in  this  regard.  Multiple  attempts  were  made  to  engage  schools  to  submit 
proposals  for  these  late  projects  without  success. 

1.  Project  Proposal  Solicitation  (May-  July) 

—  Sponsors  submit  proposals  to  Marketplace  (with  help  from  Marketplace 
administrators). 

—  Faculty/students  are  able  to  submit  proposals  to  the  CM  portal. 

2.  Bidding  by  Student  Teams  (May  -  August) 

—  Faculty  and  students  browse  marketplace  and  make  bids  on  projects  (with  help 
from  Marketplace  administrators). 

3.  Matching  of  Teams  to  Projects  (May  -  August) 

—  Sponsors  select  teams  (with  help  from  Marketplace  administrators). 
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4.  Execution  of  Projects  (September  -  May) 


—  Student  teams  work  on  projects  under  supervision  of  faculty. 

—  Sponsors  advise  and  review  work  (suggestion  of  checking  in/phone  call  one  to 
two  times  per  month  with  faculty  and/or  students). 

5.  Feedback  from  Sponsors,  Students  and  Faculty  (May-June) 

—  Marketplace  administrators  collect  feedback  from  all  participants  to  use  in 
improving  advice  in  future  years. 

Information  and  resources  are  able  to  be  added  to  the  website  to  better  enable  participating 
students  and  faculty  to  implement  SE  processes  and  tools  and  to  provide  guidelines  for  topics 
such  as  intellectual  property,  disclosure  of  project  information,  publication  of  research  results, 
and  communication  protocols.  The  guidelines  were  intended  to  reduce  some  of  the 
complications  associated  with  the  collaboration  between  undergraduate  academic  programs 
and  US  DoD  sponsors  with  respect  to  procedures  and  processes  that  arose  in  previous  Capstone 
Marketplace  projects.  In  2014-15  several  issues  regarding  disclosure  of  information  and 
communication  breakdowns  continued  to  occur,  though  efforts  were  made  to  remove.  In  2015- 
2016,  we  implemented  team  meetings  with  the  Executive  Director  of  Sponsored  Programs 
(Stevens),  Executive  Director  of  Intellectual  Property  (Stevens),  the  SERC  Program  Manager  of 
Operations,  and  the  Principal  Investigator  of  Capstone  Marketplace  to  walk  through  Intellectual 
Property,  Fabricated  Equipment,  deliverable  markings,  and  information  flow  (see  appendices 
for  copies  made  available  to  each  team). 

Moving  forward,  we  also  suggest  implementing  some  other  cross-cutting  orthogonal  activities 
across  the  CM. 
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Table  5.  Cross-Cutting  Orthogonal  Activities  Table 


WHY  -  Objectives 


WHAT  -  Initiatives 


WHY  -  Objectives 

WHAT  -  Initiatives 

Inform  students  on  appropriate  decorum  while  working  with  the  SERC:  the 
applicable  Intellectual  Property  rights  as  stated  by  FAR  and  DFAR,  proper  markings  on 
all  publications/presentations/deliverables  through  contract,  and  fabricated 
equipment. 

This  was  initiated  last  year,  but  the  objective  would  be  to  make  it  a  more  formal 
process.  This  is  to  ensure  that  all  the  end  deliverables  and  objectives  are  met 
with  no  surprises  and  students  understand  the  expectations  and  FAR/DFAR 
aspects  of  the  project  as  well  as  the  aspect  of  sensitivity,  should  any  arise. 

All  senior  design  teams  will  gothrough  a 

1  hour  seminar  on  Intellectual  Property 
rights,  deliverable  markings,  and  related 
topics 

To  ensure  students  are  gaining  knowledge  on  SE  and  SE  design.  This  will  be  done  in  a 
manner  that  translates  SE  principles  into  a  set  of  simple,  reflective  questions  in  a 
language  that  is  very  practical  —  Selected  questions  are  included  in  the  attachment. 
One  of  the  ideas  of  the  Capstone  was  to  expose  undergraduate  students  to 
systems  engineering  (overview,  principles,  methods,  processes,  and  tools)  with 
the  idea  of  enabling  them  to  start  understanding  systems  thinking  and  equip  them 
to  transition  into  the  workforce  with  greater  a bility  to  impact  whatever  program 
they  are  designated. 

Senior  designteams  will  gothrough  three 

1  hour  webinars  on  systems  engineering 
and  design.  Once  developed,  these 
webinars  will  be  posted  on  the  SERC 
website  as  a  resource  for  other  students. 

To  have  students  apply  what  is  learned.  It  also  will  provide  insight  to  newer  tools 
being  used  by  undergraduate  students,  and  help  create  a  larger  repository  of 
impressions  for  analysis  by  the  sponsors. 

All  senior  design  teams  will  be  given 
access  to  a  systems  modeling  tool  and 
requested  to  develop  the  requirements 
and  architecture  models  for  their  systems 
in  a  model  centric  environment 

Continuously  improve  the  Capstone  Marketplace,  understand  the  true  impact  SE  and 
systems  thinking  had  on  the  students,  and  to  gauge  the  influence  of  modeling  on  the 
projects  and  students.  This  will  also  serve  as  a  verification/validation  on  the  survey 

results. 

All  design  teams  will  develop  a  reflective 
paper  on  the  influence  of  systems 
engineering  and  thinking  on  their 
projects;  the  influence  of  the  modeling 
tools  —  and  thoughts  on  how  to  improve 
this  experience 

To  track  students  —  the  baseline,  progress,  final  stages,  and  where  the  student  is 
placed  in  the  workforce.  Another  objective  is  to  show  continuing  credibility  to  the 
foundational  research  that  the  multidisciplinary  teams  are  beneficial  to  the  students 
appreciation  and  use  of  systems  thinking.  Another  objective  is  to  learn  from  the 
students  —  to  see  what  new,  hot  tools  the  students  are  using  to  develop  their 
models  to  produce  the  products  and  deliverables.  This  can  help  sponsors  stay 
abreast  of  new,  upcoming  tools. 

1. Students  will  be  given  a  before  and 
after  survey  to  assess  their  knowledge  of 
systems  engineering,  model-centric 
engineering,  and  to  gauge  their  methods, 
processes,  and  tools  used  prior  and 
during  the  project.  The  end  survey  will 
also  inquire  if  the  student  is  going  into 
government,  academia,  or  industry. 

Create  excitement  and  opportunity  amongstteams  sponsored  through  the  Capstone 
Marketplace,  and  the  opportunity  to  present  at  NDIA  which  would  further  expose 
the  students  to  key  personnel  Defense  Industry  and  Defense  Industrial  Base. 

Evaluate  the  sponsored  senior  design 
projects,  help  identify  the  winning  team  — 
this  team  will  get  a  chance  to  present 
their  project  at  the  NDIA  symposium  the 
following  yearand  be  recognized  in  this 

forum. 

To  promote  the  Capstone  marketplace  to  potential  sponsors,  universities,  and  ideal 
students. 

1.  Necessaryto  engage  outstanding  students  as  soon  as  possible,  and  the 
exposure  of  students  participating  on  the  projects  could  entice  more  to 
participate. 

2.  This  would  also  help  usher  more  universities  within  the  consortium  to 
participate.  Having  the  abilityto  show  something  substantial,  such  as  a 
picture  or  interesting  update,  makes  the  projects  palpable. 

3.  For  potential  sponsors,  they  can  see  how  engaged  the  students  are. 

Social  media  outlets  (Twitter,  Facebook, 
etc.)  will  be  made  availableforstudents 
to  post  pictures  of  progress,  teamwork, 
and  final  projects. 

Teams  will  be  briefed  on  proper 
markings  to  show  sponsorship  from 

SERCand  any  others  (i.e.,  USSOCOM) 
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An  example  of  key  systems  engineering  questions  for  the  students  is  in  the  appendix.  It  would 
be  beneficial  to  meet  with  the  USSOCOM  sponsors  for  a  one-day  workshop  on  initiating  the 
timeline,  orthogonal  activities,  and  further  development  of  the  website  and  accommodations 
necessary  for  full  ability  to  view  and  be  involved  in  the  project.  This  would  also  enable  the 
foundation  to  be  solidified  and  then  in  the  future,  further  refined  and  developed. 


Website 

A  new  website  was  developed  and  launched  through  this  research  with  a  several  key  goals; 
enabling  the  scaling  of  participation  without  creating  excess  administrative  needs,  enabling  the 
transition  of  the  marketplace  from  within  the  SERC  to  another  independent  entity  or  as  a  stand¬ 
alone  service,  and  to  improve  the  user  experience  and  better  establish  the  website  as  a  project 
resource.  This  is,  however,  still  being  updated  by  pulling  over  information  from  the  old  website 
as  well  as  building  out  sections  and  testing  areas  with  outside  participants. 

The  list  below  identifies  the  new  capabilities  sought  after  from  the  foundation  previously  set, 
and  which  were  successful  in  implementing/setting  up  for  full  launch: 

•  Team  formation: 

o  Searchable  key  topics  database  that  will  tie  together: 

■  Project  requirements  as  defined  by  the  sponsor, 

■  Areas  of  expertise  or  interest  as  defined  by  mentors  and  faculty, 

■  Areas  of  interest,  experience  and  academic  disciplines  as  defined  by  the 
students. 

■  This  will  enable  identification  of  synergies  between  all  users  with  the  goal 
of  better  enabling  multiple  institution  collaboration  and  participation  of 
non-sponsor  mentors  or  SMEs. 

•  The  above  has  all  been  made  available  through  the  new  platform, 
but  does  need  to  be  built  out  more  specifically.  Previously,  the 
search  function  did  not  allow  for  pulling  specific  phrases  or  areas 
of  interest,  however,  that  has  been  improved. 

o  A  message  board  to  allow  for  communication  at  the  preliminary  team  forming 
stages  where  potential  students  can  establish  a  line  of  communication  the  same 
way  students  in  a  single  institution  may  communicate  amongst  themselves  to 
team  form  based  on  interest  and  capabilities. 

■  This  has  been  developed,  but  we  do  need  to  test. 

•  Project  support: 

o  The  resources  section  of  the  website  will  continue  to  be  expanded 

■  A  tool  for  securely  storing  relevant  project  materials  will  be  added  to 
enable  sharing  across  institutions,  sponsors,  mentors,  and  if  applicable 
the  public  viewers  of  the  website. 

■  A  shared  project  calendar  permitting  transition  to  popular  web  based 
calendar  tools. 

•  The  securing  project  materials  has  been  added,  however,  the 
shared  project  calendar  still  needs  to  be  implemented. 
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•  Assessment: 

o  Core  systems  engineering  principles  are  being  established  and  a  pre-project  and 
post  project  assessment  will  be  developed  to  better  enable  an  assessment  of 
student  learning  and  the  efficacy  of  the  Capstone  Marketplace. 

■  As  specified  in  the  cross-cutting  orthogonal  activities,  core  activities  have 
been  identified  to  view  and  share  progress  of  teams  and  their 
understanding  of  systems  engineering  principles. 


Outreach 


Soliciting  Project  Proposals  from  Sponsors 

The  SERC  team  met  with  potential  sponsors  to  provide  information  on  the  Capstone 
Marketplace.  This  ranged  from  presentations  and  in-depth  discussions  of  information  at 
system  engineering  industry  events,  to  discussions/networking  with  contacts  developed 
through  existing  professional  relationships.  The  enthusiasm  to  get  involved  is  certainly  present, 
and  many  meetings  lead  to  in-depth  discussions  and  follow  ups. 

Below  is  a  summary  of  some  of  the  key  events  attended  and  groups  contacted  and/or  briefed 
on  the  Capstone  Marketplace: 

•  NDIA  Systems  Engineering  Conference 

•  INCOSE  International  Symposium 

•  INCOSE  Interworking  Group  (January  Meeting) 

•  NOAA/NWS 

•  SE  Forum 

•  NDIA  SE  Division  Meeting 

•  Office  of  the  Science,  Technology,  Engineering,  &  Mathematics  (STEMS)  Development 
Program  within  the  Department  of  Defense 

•  Undergraduate  IWG  Knowledge  Exchange  Summit 

•  Scholastic 

•  MITRE 

•  Boeing 

•  Booz  Allen-Hamilton 

•  Differing  SYSCOMS  that  could  be  involved 

MITRE  has  stated  that  they  would  pursue  doing  2  CM  projects  per  academic  year. 


Soliciting  and  Encouraging  Academic  Participation 

A  briefing  of  Capstone  Marketplace  was  given  at  the  SERC  Executive  Advisory  Board  with  a 
majority  of  the  collaborating  universities'  Vice  Provost  of  Research  present.  This  helped  engage 
key  members  within  the  consortium.  Along  with  those,  we  also  sent  an  email  to  each  of  the 
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Deans  of  Engineering.  From  the  email,  we  received  positive  feedback  for  future  collaboration, 
and  have  further  discussed  with  participating  universities  their  involvement.  Briefings  during 
SERC  Collaborator  WebEx's  were  provided,  as  well  as  participation  in  academic  forums  that 
were  focused  on  capstone  projects,  for  instance  at  the  Conference  on  Systems  Engineering 
(CSER)  and  the  Annual  SERC  Sponsored  Research  Review  (SSRR).  Of  course,  additional  outreach 
to  individual  institutions  and  faculty  members  was  ongoing  throughout  the  project,  and  a  much 
better  understanding  has  been  established.  The  significant  perception  that  participation  in  the 
CM  will  require  excessive  administrative  effort  and  reporting  has  been  thwarted  through 
conversation  and  examples  of  execution,  but  an  explanation  should  be  included  in  the  website 
specific  to  each  project. 

SERC  Leadership  also  went  to  understand  the  relationship  between  CM  and  the  Stanford 
Hacking  for  Defense  Initiative  and  to  explore  the  intersection  of  systems  engineering  and 
innovation. 

Lessons  Learned  and  Recommendations 


Aligning  Sponsor  Involvement  with  Academic  Calendar 

The  alignment  of  the  project  solicitation  to  selection  and  execution  is  essential  to  project 
success.  Identifying  and  publishing  desired  projects  to  the  website  between  March  and  July 
would  be  ideal  for  curating  the  best  students.  It  also  helps  engage  the  academic  community, 
such  as  the  faculty.  Deans  of  Engineering,  and  Vice  Provosts  of  Research.  The  information  is 
more  widely  acknowledged  if  done  during  the  spring  to  early  summer  semesters.  The  selection 
of  the  projects  prior  to  the  start  of  the  fall  semester  would  be  beneficial  for  the  students  since 
they  often  have  to  start  working  on  their  projects  in  the  fall. 

The  role  of  the  sponsor's  subject  matter  expert  (SME)  is  crucial  to  driving  a  successful  project, 
and  the  earlier  the  SME  can  be  involved,  the  better.  The  aforementioned  timelines  would  be 
advantageous  to  USSOCOM  finding  early  involvement  from  SMEs  as  well,  but  it  is  necessary  to 
obtain  which  projects  should  be  selected  for  funding  as  early  as  possible,  so  that  the  teams  may 
be  alerted.  The  projects  are  run  on  a  short  timeline  with  the  need  of  critical  information  early. 
Of  course,  additional  reviewers  and  mentors  can  be  found  by  contacting  local  chapters  of 
professional  societies  or  personal  contacts.  For  instance,  one  student  in  spring  2017  was  able 
to  interview  a  relative  who  was  a  former  Navy  SEAL  to  help  understand  the  need,  outlook,  and 
requirements  for  the  stakeholder. 

The  SERC  ended  up  exploring  the  option  of  deobligating  funds  to  reobligate  or  to  utilize  a  NCE 
to  help  with  the  following  year's  CM  projects.  Only  5  projects  were  designated  for  the  2015- 
2016  academic  year,  with  an  additional  project  over  the  summer,  and  in  hopes  of  fulfilling  more 
projects  as  the  academic  year  and  summer  progressed,  the  funds  were  held.  The  small  number 
of  funded  projects  came  from  the  unknown  of  whether  it  would  be  selected  for  funding,  and 
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the  timeframe  of  the  initiation  with  the  sponsor.  Students  and  faculty  try  to  obtain  the  project 
information,  including  funding  process,  by  early  fall  semester. 

For  2015-2016  academic  year,  there  was  immediate  awareness  of  a  potential  deobligation  or 
NCE  if  the  remaining  projects  could  not  be  assigned.  The  deficit  comes  not  only  from  the  lack  of 
projects,  but  also  the  fact  that  some  projects  do  not  need  the  full  $10,000  delegated,  and  the 
lack  of  charging  the  expected  personnel  to  help  manage  the  Capstone  Marketplace  impacted 
the  bottom  line.  However,  aligning  the  go-ahead  on  projects  more  with  the  academic  calendar 
would  allow  the  sponsor  and  CM  PI  insight  into  the  budget  necessary  for  each  project. 
Therefore,  the  funding  can  be  adjusted  and  appropriately  allocated  to  the  sponsor  needs. 


Faculty  and  Student  Involvement 

The  faculty  advisors  much  be  able  to  participate  and  disseminate  information  to  their 
respective  design  teams.  There  is  minimal  burden  added  to  them,  however,  the  advisors  still 
need  to  oversee  their  work  and  provide  guidance  while  being  able  to  advise  sponsors  of 
realistic  expectations.  Utilizing  the  discussions  forum  through  the  website  would  enable  faculty 
to  oversee  communications  and  act  as  necessary. 

Student  involvement  is  the  most  critical  component  of  the  CM.  The  institution  of  a  friendly 
competitive  atmosphere  through  opportunity  to  have  a  "winning"  team  present  their  project  at 
the  NDIA  symposium  the  following  year  would  help  cultivate  excitement.  Also,  the  creation  of 
social  media  outlets  (Instagram,  Twitter,  Facebook,  and  LinkedIN  group)  would  not  only 
promote  the  Capstone  to  potential  sponsors  and  universities,  but  also  engage  outstanding 
students  as  soon  as  possible.  Students  like  to  showcase  their  work,  and  this  provides  the 
means  to  do  so.  An  Instagram  and  Twitter  have  been  initiated  (both  have  usernames 
SERCCapstone),  though  they  have  not  yet  been  used.  Part  of  this,  is  they  need  to  be  promoted 
throughout  the  lifecycle  of  the  project  and  presented  as  a  part  of  the  experience.  These  were 
initiated  in  the  midst  and  towards  the  end  of  the  project  lifecycle.  This  would  be  something  the 
sponsor  and  PI  could  work  on  establishing  together  (i.e.,  information  to  include  such  past 
projects  that  are  desirable  for  promotion,  etc.). 


Foundation  for  Sustained  Growth 

As  we  have  moved  forward  with  running  the  CM,  the  need  for  structure  and  a  platform  for 
sustained  growth  became  evident,  especially  if  the  2019  CM  Objective  (shown  below)  will  be 
achieved.  The  vision  for  the  Capstone  was  to  have  as  many  of  the  collaborating  universities 
involved  with  numerous  projects  and  sponsors  involved.  There  has  already  been  great 
feedback  from  universities  on  how  they  have  developed  their  capstone  courses  to  be  more 
interdisciplinary  and  open  to  outside  involvement  and  guidance.  During  this  task,  Michael 
Pennotti  helped  evaluate  the  current  structure  and  help  guide  an  architecture  for  success  that 
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is  scalable.  His  report  was  that  he  found  widespread  enthusiasm  for  the  concept,  but  also 
widespread  frustration  with  the  ever-changing  points  of  contact  and  significant  obstacles  with 
the  DoD  funding  cycles  and  academic  calendars.  His  recommendation  is  to  have  a  leader  with 
the  ability  to  take  a  greater  and  more  concentrated  effort  than  what  has  been  available  in  the 
past. 


2019  CAPSTONE  MARKETPLACE  OBJECTIVE 

The  Capstone  Marketplace  has  transformed  how  thousands  of  engineering  students  in  universities 
worldwide  conduct  their  capstone  project,  integrating  systems  engineering,  multiple  disciplines,  and 
systems  thinking  into  projects  supported  by  a  wide  array  of  private  and  public  organizations. 

STRATCG  CS  INITIATIVCS  201 9  TARGCT  METRICS 


1.  Build  brand  awareness 
among  universities  and 
potential  project 
sponsors 

2.  Build  alliances  with 
professional  societies 
and  accreditation 
organizations 

3.  Establish  Infrastructure 
and  effective 
governance  to  support 
large-scale  operation  of 
the  Capstone 
Marketplace 

4.  Lead  by  example 
through  robust 
participation  by  SERC 
Collaborators 

5.  Build  an  active 
community  of 
universities  and 
sponsors 


Partner:  Initially  partner  witn  INCOSE.  NDlA  SE 
Division,  an  :  ASEE  to  involve  both  tneir  industrial  anc 
academic  members,  eventua  l/  reach  ng  to  other 
sooetes  both  n  me  US  and  elsewhere:  establish  an 
acbve  Marketplace  community  with  individual  un  vers' ties 
and  sponsors 

| - 

Transition:  Establish  shared  governance  between 

SERC  and  INCOSE  and  perhaps  others  to  operate  and 
manage  the  Capstone  Marketpace  for  the  benefit  of  Ihe 
worldwide  engineering  community 

Outreach:  Hold  mformabonal  events,  publ  sh  success 
stores,  participate  in  capstone  anc  related  conferences 
and  take  other  steps  separately  and  jointly  with  our 
partners  to  create  broad  awareness  of  the  value  of  the 
Marketoace 


Lead  by  Example:  Encou'age  all  SERC  Collaborators 
to  partcipate  m  the  Capstone  Marketplace  and  to 
encourage  companies  and  government  agenoes  in  their 
netwo'ks  to  sponsor  Marketplace  projects 

Scale  Infrastructure:  Scale  the  website  and  supponng 
infrastructure,  ensure  both  SERC  and  partner  funding 
and  volunteer  labor,  and  establish  agreements  to  protect 
intellectual  property 


Influence  Accreditation:  Work  witn  ABET  and  other 
accreditation  organizations  to  mfuence  requ  rements  for 
capstone  projects  to  include  systems  engineering, 
multiple  disc  plmes  and  systems  thinking 


500  Marketplace  protects  sponsored  by  2C0 
organizations  in  100  universities  m  6 counties,  all 
belonging  to  a  vibrant  Capstone  Marketplace 
Community 


Robust  an  ances  between  SERC.  INCOSE.  NOIA 
SE  Division.  ASEE.  and  3  other  natonal 
eng  neerng  education  sooetes 


SERC  and  INCOSE  (and  perhaps  other  societies) 
jointly  govern  the  Marketplace 


Annual  presentations  and  informational  events  at 
each  ally  major  conference,  and  an  annual  artcle  in 
each  ally's  appropriate  publication 


At  least  75  INCOSE  and  NDIA  SE  Division 
corporate  and  government  members  participate  n 
the  Marketplace:  80VI  of  INCOSE  Academic 
Council  members  participate  m  the  Marketplace 


ABET  and  at  least  one  other  national  accreditation 
organization  incorporate  systems  engineering, 
multiple  disciplines  and  systems  thinking  into  tneir 
accreditation  requirements  for  capstone  projects 


Figure  1.  2019  Capstone  Marketplace  Objective 


Conclusions 


Overall,  there  is  tremendous  interest  in  the  topic,  and  the  CM  continues  to  be  viewed  as  a 
success  by  the  participants  across  the  three  groups:  academic,  sponsor,  and  mentor.  However, 
there  needs  to  be  a  more  robust  and  consistent  approach  from  the  leadership  to  realize  its 
potential,  and  create  a  foundation  to  then  sustain  it. 

Aligning  discussions  and  decisions  between  the  DoD  and  academic  calendars  will  be  pivotal  to 
the  success  of  the  Marketplace.  Enabling  that  open,  consistent  line  of  communication  and 
demonstration  of  project  progress  will  help  ignite  interest  and  participation  as  it  continues.  The 
growth  of  the  Capstone  into  a  year-round  discussion  with  several  ongoing  and  potential 
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projects  is  a  goal.  For  instance,  the  Auburn  task  was  started  late,  however,  it  was  a  success  for 
a  quick  turnaround  on  contract,  and  also  demonstrated  the  power  of  discussion  with  potential 
smaller  projects  that  would  be  of  interest  to  the  sponsor. 

Increasing  academic  participation  may  be  a  challenge  with  schools  who  have  a  strong,  existing 
relationship  with  industry  partners.  However,  several  lessons  learned  regarding  opportunities 
and  challenges  to  broader  participation  are  noted  in  this  report,  and  once  a  foundation  is  set,  it 
would  be  worth  pursuing  multi-school  collaboration. 

The  primary  sponsor  of  CM  projects  has  expressed  a  desire  for  them  to  continue,  and  have 
provided  beyond  excellent  SMEs.  Almost  every  faculty  member  and  student  has  stated  that  the 
SMEs  approached  participation  from  a  great  perspective  of  steering  students  but  not  directing 
them  towards  a  given  solution.  In  several  instances  this  approach  has  led  to  solutions  that  have 
been  moved  forward  for  additional  research.  It  would  be  wonderful  to  have  a  workshop  with 
the  primary  sponsor,  USSOCOM,  if  we  do  indeed  continue  to  discuss  schedule,  potential 
projects  (both  proposed  by  USSOCOM  and  the  SERC  Collaborating  universities),  website,  and 
social  media  platforms.  We  should  also  find  an  additional  participating  faculty  member  with 
the  availability  to  have  a  greater,  more  concentrated  effort  than  previously  done. 
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Appendix 


Stevens(lnstitute(of(Technology( 

http://www.sercuarc.org/(( 

i 


"STU  D  E  NT(R  ES  E  A  RCH  (P  RO  J  ECTS(( 
and( 

INTELLECTUAL(PROPERTY"( 

( 

l.((  GENERAL(INFORMATION(( 

Student  !research!opportunities!are  !a  !university!objective!for!those  (students  !interested!in!the!research!process  land! 
associated  Entrepreneurial  Experience.  !!While!all!research!does!not!automatically  Head  Itolthe  development  !of! 
intellectual  [property,  lit'sla  Icircumstancelthatldoesloccur.llThis document lislcreatedltoladdresslthatlcircumstance land  I 
ensure  Ithatlstudent  [researchers,  Ithelfacultyladvisorlandlthe  (university's  linterestslare  lall  [protected  land  Isoundly  I 
managed.!!! 


II.  I  UNIVERSITY("PATENT(POLICY"( 

ThelUniversity's  ("Patent  !Policy"!is!found  Ion  IthelStevens'  Iwebpages  land  describes  lall  (aspects  loflownership  (associated  I 
with  (research,  (sponsored  (and  (unsponsored  (and  (whether (the (inventor (is (a  (faculty, (staff  (or (student. !!! 

! 

I I I .  (((  STUDENT(RESEARCH( 

Specifics  (regarding  (the  (rights  (of  (inventorship  (as  (it  (regards  (students  (islfound  (at  !III(E)(1,  (2  (and  (3)  (in  (the  ("Patent  (Policy."  !!ln! 
order  (to  Ensure  (correct  (assignment  (of  (rights  (and  (responsibilities,  (all  (student  (inventors  ((and  (their  (faculty  (advisors,  (if! 
applicable)  (should  (contact  !the!Stevens'  (Office  (of  (Innovation  (and  !Entrepreneurship!as!soon!as!it!is!thought!that! 
intellectual  (property  (has  (been  developed.  ((This  (will  (help  (all  (affected  (parties  (and  decrease  (the  (opportunity  (for! 
misunderstanding.! 


IV.(  SERC(CAPSTONE(MARKETPLACE(PROJECTS( 

Capstone  (Marketplace  (projects  (are  (undertaken  (by  (student  (teams  (at !va  rious  (universities  (within  (the  (SERCICollaboration. !! 
The  (projects  (are  Engaged  (to  (satisfy  (the  (needs  (of  (government  (sponsors  !and!further!the!government's!interest!in!and! 
support  (of  (the  (general  Education  (and  (research  (process.  ((Sometimes  (the  (projects  (will  (result  (in  (the  (generation  (of! 
intellectual  (property  (and  (it  (is  !i  m  po  rta  nt  (that  (all  (individuals  (involved  (in  (the  (projects  (clearly  (understand  (their  (rights  (as  (well! 
as  (their  (obligations  (to  (the  (government.  !!CAPSTONE(projects(issued(by(the(SIT(SERC(are(federally(sponsored(awards(and( 
subject(to(the(contract(obligations(of(relevant(FARs(and(DFARs. ((Additional  (information  (regarding  (CAPSTONE  (Projects  !is! 
available  (in  (the  document  Entitled  ("CAPSTONEIPROJECTSHRelease  (of  (Information  (and  (Intellectual  (Property."!!! 


SUMMARYIOF  (DISCUSSION:! 


Signature  (and  date  (of  (Student  (Team  (Member:! 

! 

Signature  (and  date  (of  (Faculty  (Advisor:! _ 

! 
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SYSTEMS  ENGINEERING 
Research  Center  !! 


StevensInstitute’bf’Technology* 

http://www.sercuarc.org/* 

* 


"CAPSTONEI’ROJECTS51* 

RELEASEt)FlNFORMATION'ANDlNTELLECTUALl>ROPERTY''* 

* 

I. *  GENERALlNFORMATION* 

Capstone!Marketplace!projects!are!undertaken!by  !student!teams!within!the!SERC!Collaboration,!as!well!as!other! 
participating!universities.!rrhe!projects!are!engaged!to!satisfy!the!needs!of!government!sponsors!and!further!the! 
government's  linterestlin  land  !support!of!the!general!education  land  !research!process.!!Sometimes!the!projects!will!result! 
in!the!generation!of!intellectual!property  land  lit  !is limportant  Ithat  lall  lindivid uals linvolved  !in  !the  Iprojects Idea rly ! 
understandltheirlrights  laslwell  !as!their!obligations!to!the!government.!!CAPSTONE’l[jrojects1ssuedl3y’lthe$ITaSERC*hre* 
federally’tponsored%wards%nd%ubject1:o1;he1;ontractt)bligations’bf1-elevantiFARs!cind1DFARs.* 

II. *  PO I  NTS  t)  F  tO  NT  ACT* 

Each  !project!should!have!a!Faculty 'Advisor  !who!will!provide!guida  nee  !to!the!student!team.!!Each!project!sponsor! 
should  !identify!a!point!of!contact  !in!their!organization!who!will!provide!assistance!to!the!team!and!act  AsIthelClientTor! 
the  Iproject.  !!!Points  !of!contact!will!primarily!be!between!the!Faculty!Advisor,!the!student!team  land  Ithe! 
Client/government!sponsor;!however,  Iresponsibilitylforlreportingltheldevelopmentloflintellectuallproperty  developed  las 
the  Iresult  lot  ICAPSTON  E  Isponsorship  Irestslwith  Ithe  IStevens'lSERC.I 

III. *  DISCLOSURE'AND'RELEASE'OF'PROJECT'lNFORMATION* 

The  loverwhel  minglmajority  lofIResearch  Tasks !( RTs)  lissued  Iby  Ithe  ISERCIarelfundamentallresearch  land  Ithe  la  ward  I 
recipients  Imaylpublishlwithoutlsponsorlreview  lor  la  pproval.HProjects  Ithat  Ido  lhave  Ipublication  Irestrictions  Iwill  Ibe  Idea  rly  I 
identified!and!SERC!Collaboration!universities!may!choose!not!to!accept!these!projects.!!Since!government!requirement! 
of  Iprior  Ireview  land  la  pproval  Icould  Iresult  Ithe  Idelay  lor  Irestriction  lot  Ithesis  lor  Idissertation  Imaterial,  lit  lis  limportant  It  hat  I 
CAPSTONEIstudentHeamsIbelapprisedtoflthisIcircumstance.l 

IV. *  INTELLECTUALAROPERTY* 

At  la  Imini  mum,  development  loflintellectuallproperty  IduringlalCAPSTONElexperiencelisIsubjectltolthelrequirementslof! 
FAR!52.227Cll!and!possibly!252.227C7013/!which!require!that!the!SERC!report!all!such!intellectual!property!to!the! 
government  land  Ithat  Ithe  Igovernment  Ireceive  la  Ilicense.lllAppendix  IF  !of!the!"Sponsored  Research  lAgreementlQI  PI 
Provisions"!recognizes!the!authority  lof  leach  luniversity's  town  IPatentIPolicy  las  lit  Iregards Assignment  loflinventorship land  I 
copyright;!however!each!university's!Patent!Policy  landlprocesseslmust  Ibe  la  ble  Ito  Accommodate  Ithe  Irequirements  lof! 
reporting  And  Ithe  Assign  ment  lof  (licenses  !to  Ithe  Igovernment  Iby  Imea  ns  loflContractor/Subcontractorlownership  lof  IIP.  Ill  n  I 
the  levent  Ithe  lintellectua  I  Iproperty  lis  Ideveloped  Iby  lalcollaborating  Institution,  Ithe  Signature  lof  Ithe  Authorized  toff  iciallat  I 
Cla  use  13  llin  Ithe  ISubcontract/SponsoredIResearch  (Agreement  Acknowledges  [understanding  And  Agreement  Iwithlthese  I 
requirements.  IThelfol  lowing  (process  (describes  Ithe  tool  la  boratinglinstitution'slinteractionlwith  Ithe  ISERC:  I 

•  !  DEVELOPEDATtOLLABORATIONUNIVERSITYEIFacultylAdvisorlnotifiesItheirluniversitylbusinessloffice  laslwell! 

as  Ithe  ISE  RC  IMa  nager  lof  IProgra  m  lOperations,  Iwho  Iwill  Inotify  It  he  IStevens  (Office  lof  lln  novation  And! 
Entrepreneurship!(SEE!APPENDIX!F!-!IP  [PROVISIONS)  I 

•  !  DEVELOPEDAT’STEVENSEIFacultylAdvisorlnotifiesISERCIManagerloflProgramlOperations,  Iwho  Iwill  Inotify  Ithe  I 

Stevens  [Office  lof  linn  ovation  land  !Entrepreneurship!(SEE  [APPENDIX  IF  Ell  P  [PROVISIONS)  I 

! 

The  following  [section  Advises  Ithe  Student  Team  [members,  [the  [Faculty  [Advisor  And  Ithe  [Collaboration  !U  niversity  !(if! 
relevant)  lof  Ipublication  Irestrictions.  [[Additionally,  [each  [member  lof  Ithe  [Student  Team  And  Ithe  [Faculty  [Advisory  I 
acknowledgeslreceiptloflthislinformationlbylmeanstofltheirlsignaturelonlthislmemo.l 
! 

AreltherelpublicationlrestrictionstonlRT!#! _ !?!  I  I  YES!  I  NO! 

! 

Signaturelandldate  lof  Student  Team  [Member:! _ ! 

! 

Signature  And  Idate  lof  [Faculty  lAdvisor: ! _ I 
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SYSTEMS  ENGINEERING 
Research  Center  !! 


Stevens%istitute%f%echnology% 

http://www.sercuarc.org/%c1 

o/ 


SERC%R0CESS%/IEM0:%  "SERC%APSTONE%Ts%nd%abricated%quipment%s.®eliverables"% 

Sometimes!the!difference!between!equipment,!fabricated!equipment!and!a!project  (deliverable  Irequires  Iclarification. !! 
This!PROCESS!MEMO!seeks!to!provide!guidelines!to!the!SERC!research!community!in!this!regard.! 

Typically,  la  Ideliverablelis  Ian  litem  loflequipment lor IscientificlinstrumentationlcreatedlbylStevens'lpersonnellin  Icon nection  I 
with  la  Sponsored  Iproject  Ho  Iwhich  IStevens  Iwill  Inot  Ihold  Ititle.  IIAdditionally,  lit  lis  loften  Ithe  lease  Ithat  Ithe  !c/e//ve/'o£>/e/wil  libel 
provided  Itolthelsponsor  lor  lanotherlpartylatlthe lend  lot !the!project;!although!the!sponsor lean  Idecide  Ion  lalternative  I 
dispositionloflthelitemlatlprojectCe  nd! 

lf!an!item!is!a!de//VeraWe,  lit  lean  not  IbelFabricatedlEquipment.MMerely  Icon  necting  [components  [together  linlalsystem! 
(physically  lor  Ivirtually)  Idoeslnot  [constitute  [Fabricated  lEquipment.HProjectslwhere  It  he  [construction  lot  la  Ipiecelof! 
equipmentlislthelsole  Ipurpose  lot  Ithe  [sponsored  Iproject  la  re  Inot  lei  igiblelfor  Ithe  [Fabricated  [Equipment  lexemptionlto  I 
application  lot  IF&A.NEssentially,  [Fabricated  [Equipment  lis  la  [tool  Ito  [conduct  Ithe  [research  las  [opposed  Itolbeing  Ithe  [focus  lot! 
resea  rch  !i tself.  IIAn  [explicit  [prototyping  la ctivity  lin  Ithe  Iproject  Ipla  nlwould  Inot  Ibe  [considered  Itolbe  la  Ifabricationltasklsince  I 
the  [prototype  Idoes  Inot  Iresu  It  linlalfunctiona  I  Ipiecelof  !equipment.!!Thus,!fabrication!must  Ibe  [carried  lout  [according  Ito  la  I 
plan  Ithat  Idoeslnot  [require  !experimentation!with!concepts!or!with!the!approach.!!The!development!of!the!design!is! 
consideredltolhave  [occurred  Iprior Ito Ithe  [fabrication  land  Inot  lin  Ipa rallel  Iwith  lit. !! 

Sometimes,!as!in!the!case!of!SERC!CAPSTONE!projects,!funds!are!provided!by!the!SERC!sponsors!for!the!purpose!of! 
provid  ing  [research,  Idesign  lor  [assembly  [opportunities  Ito  [undergraduate  tar  [graduate  [students  li  n  If  u  rtherancelofltheir! 
educational!experience.!!ln!these!cases!the!amount!provided!by!the!sponsor!may!exceed!the!$5, 000  [threshold  Ithat! 
generally  [describes  ['equipment;' Ihowever,  Ithe  [sponsor  !may!not  [consider  Ithis  las  [either  Ithe  [fabrication  loflequipment  lor  I 
as!a  !'de/h/erab/e.'!!Disposition!of!these  lend  Stems  Iwill  Ibe  [determined  !by  Ithe  [sponsor  las  Ithe  Iresulttof  [dialogue  [during  Ithe! 
project.!!! 
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Defining  the  Problem 
Key  Questions: 


Creating  an  Operational  Concept 

Key  Questions: 


What  operational  need  or  market  opportunity  is  your  system  intended  to 
address? 

Assessment  Criteria:  The  need  for  the  system  is  well  understood, 
fully  described  in  the  language  of  the  stakeholders  and  free  of 
solutions. 

Who  are  the  most  important  stakeholders  and  what  are  the  key 
requirements  of  each? 

Assessment  Criteria:  The  key  stakeholders  have  been  identified  and 
their  most  important  requirements  defined,  validated  and  clearly 
stated. 

What  are  the  three  to  five  most  important  features  of  your  system  that 
distinguish  it  from  those  of  your  competitors? 

Assessment  Criteria:  Features  are  specific,  quantifiable  (or  readily 
observable),  and  important  to  the  customer. 


4.  What  is  your  proposed  system  concept?  What  alternative  concepts  did  you 
consider  and  why  did  you  choose  the  one  you  proposed? 

•  Assessment  Criteria:  Broad  range  of  concepts  defined  and  systematically 
analyzed  against  criteria  linked  to  key  stakeholder  needs. 

5.  How  will  your  proposed  concept  operate  within  the  larger  context  to  achieve 
its  intended  purpose? 

•  Assessment  Criteria:  The  external  systems  with  which  the  system  will 
interact  have  been  identified,  the  system  boundary  has  been  clearly 
defined,  and  the  interactions  between  the  system  and  the  external 
systems  have  been  specified  from  a  black  box  perspective. 

6.  What  are  the  key  specifications  that  will  drive  the  system’s  design  and 
development? 

•  Assessment  Criteria:  System  requirements  a)  have  been  derived  from 
and  are  linked  to  the  stakeholder  requirements,  b)  describe  what  the 
system  shall  do  but  not  how,  and  c)  are  verifiable  and  properly  written. 


Deciding  How  the  System  Will  Work 

Key  Questions: 


What  top-level  functions  will  your  system  have  to  perform  in  order  to  accomplish 
it’s  mission? 


•  Assessment  Criteria:  Top-level  functions  ore  few  in  number  (3-6)  and  fully 
represent  the  required  system  functionality. 

8.  How  else  could  you  have  partitioned  the  system  functionality  and  why  did  you 
choose  to  partition  it  the  way  you  did? 

•  Assessment  Criteria:  Alternative  decomposition  is  plausible  and  the  rationale 
for  selecting  the  chosen  partition  is  sound. 

9.  How  will  the  top-level  functions  interact  with  each  other  and  with  their  external 
environment? 

•  Assessment  Criteria:  Key  operational  scenarios  can  be  traced  through  the  top- 
level  functions. 

to.  How  well  will  your  functional  architecture  perform  and  what  evidence  can  you 
provide  to  support  that  assessment? 

•  Assessment  Criteria:  The  functional  architecture  has  been  analyzed  to  verify 
that  it  achieves  the  key  system  requirements. 


Implementing  the  Functions 

Key  Questions: 


1 1 .  What  are  the  major  subsystems  of  your  system  and 
how  do  they  map  to  the  functions  the  system  must 
perform? 

•  Assessment  Criteria:  Subsystems  defined  and 
mapped  to  the  top-level  functional  architecture. 

12.  What  are  the  key  interfaces  between  the  subsystems 
and  between  them  and  the  external  systems  with 
which  the  system  interacts? 

•  Assessment  Criteria:  External  and  internal 
interfaces  defined  and  mapped  to  corresponding 
external  and  internal  inputs  and  outputs. 
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Specifying  Components 

Key  Questions: 

13.  What  are  the  key  requirements  for  each  of  your  major 
subsystems? 

•  Assessment  Criteria:  Verifiable  subsystem  requirements 
derived  from  key  system  requirements 

14.  What  tradeoffs  were  made  in  deriving  your  subsystem 
requirements? 

•  Assessment  Criteria:  Tradeoffs  systematically  analyzed 
against  well-defined  criteria  and  reasonable  decisions 

15.  What  are  the  most  important  technical  risks  for  your 
system  and  how  will  you  mitigate  each? 

•  Key  technical  risks  identified,  properly  characterized  as 
to  likelihood  and  consequence,  and  suitable  mitigation 
plans  provided  for  each 


Integration  and  Test 

Key  Questions: 


16.  How  will  you  integrate  and  test  the  components  of  your  system? 
Why? 

•  Assessment  Criteria:  Plan  in  place  to  systematically  test  all 
key  interfaces  and  risk  items  against  corresponding 
specifications 


17.  What  resources  and  facilities  will  you  need  to  verify  that  the 
system  meets  its  specifications? 

•  Assessment  Criteria:  Test  systems  fully  designed  and  plans  in 
place  to  ensure  they  are  available  when  needed 


Integration  and  Test 

Key  Questions: 


18.  How  will  you  prove  to  your  customer  that  the  system  meets  his/ 
her  needs  in  the  operational  environment? 

•  Assessment  Criteria:  Plans  in  place  to  demonstrate  that  the 
system  meets  the  original  customer  needs  and  to  gain  final 
customer  acceptance 

19.  How  will  you  respond  when  problems  arise? 

•  Assessment  Criteria:  Plans  and  resources  in  place  to  diagnose, 
fix  and  test  field-found  faults  and  to  certify  that  corrective 
actions  have  been  successful 
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